Skip to content

Bump @vercel/agent-eval to 2.2.1 - #97885

Merged
gaojude merged 1 commit into
canaryfrom
bump/agent-eval-2.2.1
Aug 25, 2026
Merged

Bump @vercel/agent-eval to 2.2.1#97885
gaojude merged 1 commit into
canaryfrom
bump/agent-eval-2.2.1

Conversation

@gaojude

@gaojude gaojude commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Harness bump so the eval grader rewrite in #97826 can run against the fixed judge.

@gaojude @claude
Bump @vercel/agent-eval from 1.3.0 to 2.2.1
01859c3 
Picks up the async judge matchers (vercel-labs/agent-eval#196) and the
vitest reinstall guard (vercel-labs/agent-eval#195) so judge-based evals
grade reliably.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Tests Passed

Commit: 01859c3

@github-actions

github-actions Bot commented Aug 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Stats from current PR

🔴 1 regression

Metric Canary PR Change Trend
Turbo Build Time 6.228s 6.580s 🔴 +352ms (+6%) █▇▆▄▃
📊 All Metrics
📖 Metrics Glossary

Dev Server Metrics:

  • Listen = TCP port starts accepting connections
  • First Request = HTTP server returns successful response
  • Cold = Fresh build (no cache)
  • Warm = With cached build artifacts

Build Metrics:

  • Fresh = Clean build (no .next directory)
  • Cached = With existing .next directory

Change Thresholds:

  • Time: Changes < 50ms AND < 10%, OR < 2% are insignificant
  • Size: Changes < 1KB AND < 1% are insignificant
  • All other changes are flagged to catch regressions

⚡ Dev Server

Metric Canary PR Change Trend
Cold (Listen) 865ms 865ms ✓ █████
Cold (Ready in log) 833ms 841ms ✓ ▅▅▅▄▁
Cold (First Request) 1.337s 1.367s ✓ ▄▃▇▂▁
Warm (Listen) 864ms 866ms ✓ █████
Warm (Ready in log) 834ms 858ms ✓ ▄▆▇▅▆
Warm (First Request) 1.330s 1.395s ✓ ▂▄▇▃▆
📦 Dev Server (Webpack) (Legacy)

📦 Dev Server (Webpack)

Metric Canary PR Change Trend
Cold (Listen) 814ms 864ms ✓ ▂▂▂▁▂
Cold (Ready in log) 820ms 823ms ✓ ▁▁▁▁▁
Cold (First Request) 3.531s 3.486s ✓ ▁▁▁▁▁
Warm (Listen) 864ms 864ms ✓ ▂▂▂▁▂
Warm (Ready in log) 823ms 827ms ✓ ▁▁▁▁▁
Warm (First Request) 3.563s 3.525s ✓ ▁▁▁▁▁

⚡ Production Builds

Metric Canary PR Change Trend
Fresh Build 6.228s 6.580s 🔴 +352ms (+6%) █▇▆▄▃
Cached Build 3.324s 3.346s ✓ ▅▂▆▂▁
📦 Production Builds (Webpack) (Legacy)

📦 Production Builds (Webpack)

Metric Canary PR Change Trend
Fresh Build 25.466s 25.877s ✓ ▁▁▂▁▁
Cached Build 25.789s 25.773s ✓ ▁▁▁▁▁
node_modules Size 553 MB 553 MB ✓ ▁▁▁██
📦 Bundle Sizes

Bundle Sizes

⚡ Turbopack

Client

Main Bundles
Canary PR Change
017_4moyv5m34.js gzip 117 B 117 B ✓
05jocc85-b73_.js gzip 341 B N/A -
096twxhe86hwd.js gzip 7.53 kB N/A -
0cz1d0mv5g_q7.js gzip 39.4 kB 39.4 kB ✓
0e5lgz-xcf6ln.js gzip 273 B 273 B ✓
0enhddgqs2dl-.js gzip 265 B 265 B ✓
0h34mejwa_7pd.js gzip 354 B 354 B ✓
0mg5-nyl2xcjy.js gzip 65.5 kB N/A -
0w853zxni3ro0.js gzip 1.36 kB N/A -
13lq2aw3_0bgw.js gzip 3.52 kB N/A -
17wjm_wc16a_w.js gzip 71.5 kB N/A -
1drifx6-oqf3q.js gzip 5.7 kB N/A -
1elt1qium-r2m.css gzip 115 B 115 B ✓
1m5vdnlf0deiv.js gzip 411 B 411 B ✓
1nigpy_mr0xvj.js gzip 339 B N/A -
1p0l8mn2wz5f2.js gzip 1.97 kB N/A -
1sqoyy39jpgkq.js gzip 121 B 121 B ✓
1tf1phijqlx9j.js gzip 220 B 220 B ✓
2-54v5vj7orwp.js gzip 46.8 kB N/A -
26upjmii0ixt5.js gzip 153 B N/A -
2b05_424u1zq-.js gzip 120 B N/A -
2ca40xittx95r.js gzip 8.75 kB N/A -
2kci0vnt-49o-.js gzip 113 B 113 B ✓
2rtriq9lc6zj-.js gzip 116 B N/A -
2sxgl_rcrrn_p.js gzip 399 B N/A -
2vt62i17bhbdu.js gzip 2.35 kB N/A -
342di-crrg9kq.js gzip 119 B N/A -
36hfc2_979aw2.js gzip 119 B N/A -
38wfp6jjrrdfm.js gzip 131 B 131 B ✓
3dl1ilk37-30g.js gzip 117 B N/A -
3h0us7yy7bd_o.js gzip 13.1 kB N/A -
3kvxagoobts7h.js gzip 117 B N/A -
3p99hd9vwe6ae.js gzip 13 kB N/A -
3r0475panhk6m.js gzip 123 B N/A -
3s0i-weykgzrt.js gzip 125 B N/A -
3vjosdbddykpw.js gzip 5.88 kB N/A -
3wijk2ywqgsv9.js gzip 2.52 kB N/A -
41u5s3oe2-erp.js gzip 2.29 kB N/A -
turbopack-0d..9ejy.js gzip 3.74 kB 3.74 kB ✓
02zc7sqlpqxn3.js gzip N/A 119 B -
06pzxqp5qiozm.js gzip N/A 5.7 kB -
0dug3edczk4gl.js gzip N/A 71.5 kB -
0e28eh3gbwmp0.js gzip N/A 3.51 kB -
0e53csrecof4e.js gzip N/A 65.5 kB -
0jwho9fkrb_t3.js gzip N/A 2.29 kB -
0longconn5mlm.js gzip N/A 123 B -
0p4l3ozbez2cz.js gzip N/A 2.35 kB -
0sbh_15ejop4o.js gzip N/A 13 kB -
0skdm-9rpid8g.js gzip N/A 117 B -
0wv-h56sz83wn.js gzip N/A 120 B -
137xup2t69ovc.js gzip N/A 1.97 kB -
1ezuqvubf86h1.js gzip N/A 5.88 kB -
1fk10s3m1oecx.js gzip N/A 400 B -
1hut5u-lz2yif.js gzip N/A 117 B -
20qhkyyj2_tla.js gzip N/A 116 B -
2imyufe18bsb7.js gzip N/A 1.36 kB -
2p4n3-8-xixh_.js gzip N/A 341 B -
2vqdfieehz5do.js gzip N/A 8.75 kB -
2x99jdcpr5n8x.js gzip N/A 7.53 kB -
2xn85ykzw2v1e.js gzip N/A 13.1 kB -
34knadd6wu9qg.js gzip N/A 119 B -
39jq2b6t6z6up.js gzip N/A 339 B -
3a9a9tvkag_-j.js gzip N/A 2.52 kB -
3b660b4ak0547.js gzip N/A 153 B -
3d61tgick8g6b.js gzip N/A 46.8 kB -
3t121tj-h0kbm.js gzip N/A 125 B -
Total 299 kB 299 kB ⚠️ +3 B

Server

Middleware
Canary PR Change
middleware-b..fest.js gzip 1.02 kB 1.03 kB ✓
Total 1.02 kB 1.03 kB ⚠️ +1 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 846 B 848 B ✓
Total 846 B 848 B ⚠️ +2 B
Build Cache
Canary PR Change
00000001.sst gzip 13.3 MB 12.4 MB 🟢 881 kB (-7%)
00000002.sst gzip 13.4 MB 14.2 MB 🔴 +832 kB (+6%)
00000003.sst gzip 15 MB 13.3 MB 🟢 1.69 MB (-11%)
00000004.sst gzip 13.2 MB 14.9 MB 🔴 +1.68 MB (+13%)
00000005.sst gzip 2.75 MB 10.1 MB 🔴 +7.31 MB (+265%)
00000006.sst gzip 10.1 MB 2.75 MB 🟢 7.31 MB (-73%)
00000007.sst gzip 59 B 59 B ✓
00000008.meta gzip 89 B 89 B ✓
00000009.meta gzip 297 kB 297 kB ✓
00000010.meta gzip 297 kB 297 kB ✓
00000011.meta gzip 297 kB 297 kB ✓
00000012.sst gzip 53.6 kB 52.5 kB 🟢 1.02 kB (-2%)
00000013.sst gzip 1.76 MB 1.8 MB 🔴 +40.8 kB (+2%)
00000014.sst gzip 59 B 59 B ✓
00000015.meta gzip 116 B 116 B ✓
00000016.meta gzip 327 kB 327 kB ✓
00000017.meta gzip 403 kB 404 kB ✓
00000018.sst gzip 54.3 kB 53.4 kB 🟢 915 B (-2%)
00000019.sst gzip 1.37 MB 1.36 MB 🟢 7.81 kB (-1%)
00000020.sst gzip 59 B 59 B ✓
00000021.meta gzip 116 B 116 B ✓
00000022.meta gzip 327 kB 327 kB ✓
00000023.meta gzip 371 kB 372 kB ✓
00000024.sst gzip 54.5 kB 53.4 kB 🟢 1.07 kB (-2%)
00000025.sst gzip 1.37 MB 1.36 MB 🟢 8.41 kB (-1%)
00000026.sst gzip 59 B 59 B ✓
00000027.meta gzip 116 B 116 B ✓
00000028.meta gzip 327 kB 327 kB ✓
00000029.meta gzip 371 kB 372 kB ✓
00000030.sst gzip 53.7 kB 52.6 kB 🟢 1.12 kB (-2%)
00000031.sst gzip 1.37 MB 1.36 MB 🟢 8.37 kB (-1%)
00000032.sst gzip 59 B 59 B ✓
00000033.meta gzip 116 B 116 B ✓
00000034.meta gzip 327 kB 327 kB ✓
00000035.meta gzip 371 kB 372 kB ✓
00000036.sst gzip 54.2 kB 53.2 kB 🟢 1.01 kB (-2%)
00000037.sst gzip 1.37 MB 1.36 MB 🟢 8.41 kB (-1%)
00000038.sst gzip 59 B 59 B ✓
00000039.meta gzip 116 B 116 B ✓
00000040.meta gzip 327 kB 327 kB ✓
00000041.meta gzip 371 kB 372 kB ✓
CURRENT gzip 94 B 94 B ✓
LOG gzip 670 B 679 B 🔴 +9 B (+1%)
Total 79.6 MB 79.5 MB ✅ -46.6 kB

📦 Webpack

Client

Main Bundles
Canary PR Change
3322-HASH.js gzip 65.7 kB N/A -
4191.HASH.js gzip 169 B N/A -
9784-HASH.js gzip 5.6 kB N/A -
b1ad9f4c-HASH.js gzip 63.5 kB N/A -
framework-HASH.js gzip 59.7 kB 59.7 kB ✓
main-app-HASH.js gzip 254 B 255 B ✓
main-HASH.js gzip 39.8 kB 39.7 kB ✓
webpack-HASH.js gzip 1.68 kB 1.68 kB ✓
3577.HASH.js gzip N/A 168 B -
578-HASH.js gzip N/A 65.8 kB -
8590-HASH.js gzip N/A 5.57 kB -
a8984546-HASH.js gzip N/A 63.5 kB -
Total 236 kB 237 kB ⚠️ +107 B
Polyfills
Canary PR Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB ✓
Total 39.4 kB 39.4 kB ✓
Pages
Canary PR Change
_app-HASH.js gzip 194 B 193 B ✓
_error-HASH.js gzip 181 B 182 B ✓
css-HASH.js gzip 334 B 331 B ✓
dynamic-HASH.js gzip 1.81 kB 1.81 kB ✓
edge-ssr-HASH.js gzip 255 B 253 B ✓
head-HASH.js gzip 349 B 351 B ✓
hooks-HASH.js gzip 382 B 384 B ✓
image-HASH.js gzip 5.05 kB 5.05 kB ✓
index-HASH.js gzip 260 B 259 B ✓
link-HASH.js gzip 2.48 kB 2.48 kB ✓
routerDirect..HASH.js gzip 317 B 318 B ✓
script-HASH.js gzip 384 B 386 B ✓
withRouter-HASH.js gzip 316 B 315 B ✓
1afbb74e6ecf..834.css gzip 106 B 106 B ✓
Total 12.4 kB 12.4 kB ✅ -6 B

Server

Edge SSR
Canary PR Change
edge-ssr.js gzip 128 kB 128 kB ✓
page.js gzip 293 kB 293 kB ✓
Total 421 kB 421 kB ✅ -411 B
Middleware
Canary PR Change
middleware-b..fest.js gzip 600 B 601 B ✓
middleware-r..fest.js gzip 156 B 156 B ✓
middleware.js gzip 45.6 kB 45.5 kB ✓
edge-runtime..pack.js gzip 842 B 842 B ✓
Total 47.2 kB 47.1 kB ✅ -131 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 699 B 699 B ✓
Total 699 B 699 B ✓
Build Cache
Canary PR Change
0.pack gzip 4.8 MB 4.79 MB 🟢 9.36 kB (0%)
index.pack gzip 122 kB 124 kB 🔴 +1.62 kB (+1%)
index.pack.old gzip 123 kB 125 kB 🔴 +1.93 kB (+2%)
Total 5.04 MB 5.04 MB ✅ -5.8 kB

🔄 Shared (bundler-independent)

Runtimes
Canary PR Change
app-page-exp...dev.js gzip 375 kB 375 kB ✓
app-page-exp..prod.js gzip 206 kB 206 kB ✓
app-page-tur...dev.js gzip 374 kB 374 kB ✓
app-page-tur..prod.js gzip 206 kB 206 kB ✓
app-page-tur...dev.js gzip 370 kB 370 kB ✓
app-page-tur..prod.js gzip 204 kB 204 kB ✓
app-page.run...dev.js gzip 371 kB 371 kB ✓
app-page.run..prod.js gzip 204 kB 204 kB ✓
app-route-ex...dev.js gzip 82.6 kB 82.6 kB ✓
app-route-ex..prod.js gzip 55.7 kB 55.7 kB ✓
app-route-tu...dev.js gzip 82.6 kB 82.6 kB ✓
app-route-tu..prod.js gzip 55.7 kB 55.7 kB ✓
app-route-tu...dev.js gzip 82.2 kB 82.2 kB ✓
app-route-tu..prod.js gzip 55.4 kB 55.4 kB ✓
app-route.ru...dev.js gzip 82.2 kB 82.2 kB ✓
app-route.ru..prod.js gzip 55.4 kB 55.4 kB ✓
dev-validati...dev.js gzip 134 kB 134 kB ✓
dev-validati...dev.js gzip 134 kB 134 kB ✓
dev-validati...dev.js gzip 132 kB 132 kB ✓
dev-validati...dev.js gzip 132 kB 132 kB ✓
dist_client_...dev.js gzip 324 B 324 B ✓
dist_client_...dev.js gzip 326 B 326 B ✓
dist_client_...dev.js gzip 318 B 318 B ✓
dist_client_...dev.js gzip 317 B 317 B ✓
pages-api-tu...dev.js gzip 46.3 kB 46.3 kB ✓
pages-api-tu..prod.js gzip 34.3 kB 34.3 kB ✓
pages-api.ru...dev.js gzip 46.2 kB 46.2 kB ✓
pages-api.ru..prod.js gzip 34.3 kB 34.3 kB ✓
pages-turbo....dev.js gzip 54.7 kB 54.7 kB ✓
pages-turbo...prod.js gzip 39.7 kB 39.7 kB ✓
pages.runtim...dev.js gzip 54.7 kB 54.7 kB ✓
pages.runtim..prod.js gzip 39.6 kB 39.6 kB ✓
server.runti..prod.js gzip 65.9 kB 65.9 kB ✓
use-cache-pr...dev.js gzip 73.3 kB 73.3 kB ✓
use-cache-pr...dev.js gzip 73.3 kB 73.3 kB ✓
use-cache-pr...dev.js gzip 71.5 kB 71.5 kB ✓
use-cache-pr...dev.js gzip 71.5 kB 71.5 kB ✓
Total 4.1 MB 4.1 MB ✅ -1 B
📎 Tarball URL 
https://vercel-packages.vercel.app/next/commits/01859c3ac352183aef64e960ee82ae43607d8213/next

Commit: 01859c3

@gaojude
gaojude requested a review from devjiwonchoi August 25, 2026 17:35
@gaojude
gaojude enabled auto-merge (squash) August 25, 2026 19:43
@gaojude
gaojude merged commit fcf66c3 into canary Aug 25, 2026
232 of 234 checks passed
@gaojude
gaojude deleted the bump/agent-eval-2.2.1 branch August 25, 2026 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@devjiwonchoi devjiwonchoi devjiwonchoi approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@gaojude @devjiwonchoi